Dynamic Appearance-Based Recognition

We describe a hierarchical appearance-based method for learning, recognizing, and predicting arbitrary spatiotemporal sequences of images. The method, which implements a robust hierarchical form of the Kalman filter derived from the Minimum Description Length (MDL) principle, includes as a special case several well-known object encoding techniques including eigenspace methods for static recognition. Successive levels of the hierarchical filter implement dynamic models operating over successively larger spatial and temporal scales. Each hierarchical level predicts the recognition state at a lower level and modifies its own recognition state using the residual error between the prediction and the actual lower-level state. Simultaneously, on a longer time scale, the filter learns an internal model of input dynamics by adapting its generative and state transition matrices at each level to minimize prediction errors. The resulting prediction/learning scheme thereby implements an on-line form of the well-known Expectation-Maximization (EM) algorithm from statistics. We present experimental results demonstrating the method’s efficacy in mediating robust spatiotemporal recognition in a variety of scenarios containing varying degrees of occlusions and clutter.